Oops! https://public-artifacts.taskcluster.net/RyWMqfmbRxiZJpQX8oZSnA/0/public/logs/live_backing.log [task 2017-09-05T10:34:50.647253Z] 10:34:50 INFO - Downloading packages: [u'target.common.tests.zip', u'target.mochitest.tests.zip'] for test suite category: mochitest [task 2017-09-05T10:34:50.648078Z] 10:34:50 INFO - Adding 'mozinfo.json' for extraction from common.tests zip file [task 2017-09-05T10:34:50.649126Z] 10:34:50 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip [task 2017-09-05T10:34:50.650023Z] 10:34:50 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip'}, attempt #1 [task 2017-09-05T10:34:50.650843Z] 10:34:50 INFO - Fetch https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip into memory [task 2017-09-05T10:34:52.367603Z] 10:34:52 INFO - Content-Length response header: 55022230 [task 2017-09-05T10:34:52.369495Z] 10:34:52 INFO - Bytes received: 55022230 [task 2017-09-05T10:34:53.695435Z] 10:34:53 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.mochitest.tests.zip [task 2017-09-05T10:34:53.695790Z] 10:34:53 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.mochitest.tests.zip'}, attempt #1 [task 2017-09-05T10:34:53.696053Z] 10:34:53 INFO - Fetch https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.mochitest.tests.zip into memory [task 2017-09-05T10:34:55.452903Z] 10:34:55 INFO - Content-Length response header: 76622425 [task 2017-09-05T10:34:55.453141Z] 10:34:55 INFO - Bytes received: 76622425 [task 2017-09-05T10:35:05.005066Z] 10:35:05 INFO - Downloading packages: [u'target.common.tests.zip', u'target.xpcshell.tests.zip'] for test suite category: xpcshell [task 2017-09-05T10:35:05.005446Z] 10:35:05 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip [task 2017-09-05T10:35:05.005871Z] 10:35:05 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip'}, attempt #1 [task 2017-09-05T10:35:05.006287Z] 10:35:05 INFO - Fetch https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.common.tests.zip into memory [task 2017-09-05T10:35:06.432410Z] 10:35:06 INFO - Content-Length response header: 55022230 [task 2017-09-05T10:35:06.432678Z] 10:35:06 INFO - Bytes received: 55022230 [task 2017-09-05T10:35:07.852182Z] 10:35:07 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.xpcshell.tests.zip [task 2017-09-05T10:35:07.852404Z] 10:35:07 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.xpcshell.tests.zip'}, attempt #1 [task 2017-09-05T10:35:07.852775Z] 10:35:07 INFO - Fetch https://queue.taskcluster.net/v1/task/Oz77JHY8R46lR-xvwFHp5Q/artifacts/public/build/target.xpcshell.tests.zip into memory [task 2017-09-05T10:35:12.138071Z] 10:35:12 INFO - Content-Length response header: 10574837 [task 2017-09-05T10:35:12.140379Z] 10:35:12 INFO - Bytes received: 10574837

Test packages are associated with suite categories (like 'reftests'). The test-verify task has multiple suite categories, since it can run, for instance, mochitests and xpcshell tests. The common test zip is required for both mochitests and xpcshell tests, so it is included in the list of required test packages more than once. This patch eliminates duplicates from the list before downloading, so that each package is only downloaded once. https://treeherder.mozilla.org/#/jobs?repo=try&revision=8b4f6d49479c5a5cfee71882907bd6b7cf606375&filter-tier=1&filter-tier=2&filter-tier=3 https://public-artifacts.taskcluster.net/e4W330QASByauXFQB1Qovg/0/public/logs/live_backing.log [task 2017-09-07T17:13:13.191314Z] 17:13:13 INFO - Downloading packages: [u'target.common.tests.zip', u'target.xpcshell.tests.zip', u'target.mochitest.tests.zip'] for test suite categories: ['mochitest', 'xpcshell'] [task 2017-09-07T17:13:13.193483Z] 17:13:13 INFO - Adding 'mozinfo.json' for extraction from common.tests zip file [task 2017-09-07T17:13:13.199347Z] 17:13:13 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.common.tests.zip [task 2017-09-07T17:13:13.200288Z] 17:13:13 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.common.tests.zip'}, attempt #1 [task 2017-09-07T17:13:13.200946Z] 17:13:13 INFO - Fetch https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.common.tests.zip into memory [task 2017-09-07T17:13:14.124479Z] compiz (core) - Warn: Attempted to restack relative to 0x1600006 which is not a child of the root window or a window compiz owns [task 2017-09-07T17:13:26.169086Z] 17:13:26 INFO - Content-Length response header: 51584400 [task 2017-09-07T17:13:26.170327Z] 17:13:26 INFO - Bytes received: 51584400 [task 2017-09-07T17:13:27.387171Z] 17:13:27 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.xpcshell.tests.zip [task 2017-09-07T17:13:27.388580Z] 17:13:27 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.xpcshell.tests.zip'}, attempt #1 [task 2017-09-07T17:13:27.388770Z] 17:13:27 INFO - Fetch https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.xpcshell.tests.zip into memory [task 2017-09-07T17:13:30.869397Z] 17:13:30 INFO - Content-Length response header: 13515469 [task 2017-09-07T17:13:30.870745Z] 17:13:30 INFO - Bytes received: 13515469 [task 2017-09-07T17:13:33.104032Z] 17:13:33 INFO - Downloading and extracting to /builds/worker/workspace/build/tests these dirs bin/*, certs/*, config/*, mach, marionette/*, modules/*, mozbase/*, tools/*, mochitest/*, xpcshell/*, mozinfo.json from https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.mochitest.tests.zip [task 2017-09-07T17:13:33.105303Z] 17:13:33 INFO - retry: Calling fetch_url_into_memory with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.mochitest.tests.zip'}, attempt #1 [task 2017-09-07T17:13:33.105472Z] 17:13:33 INFO - Fetch https://queue.taskcluster.net/v1/task/fNsejSg3ShS31ped28nNbw/artifacts/public/build/target.mochitest.tests.zip into memory [task 2017-09-07T17:13:39.207513Z] 17:13:39 INFO - Content-Length response header: 76741926 [task 2017-09-07T17:13:39.209332Z] 17:13:39 INFO - Bytes received: 76741926

Comment on attachment 8905593 [details] [diff] [review] eliminate duplicate packages before downloading Review of attachment 8905593 [details] [diff] [review]: ----------------------------------------------------------------- good find

Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/eef129f6e50b Update mozharness so that no test package is downloaded more than once; r=jmaher